The distance between a camera and a spatial point in a scene can be determined or well estimated from the position of the point within two or more associated images showing the same point, wherein either the scene is stationary or the associated images are captured simultaneously. The distance calculation is still possible if one or more planar mirrors are arranged in the scene, and some of the images are captured in the mirror. The three dimensional (3D) position of a point can be computed from basic geometric relationships when the relationship between the spatial position of the image recording device and the spatial position and specific parameters of the reflecting surfaces (e.g. mirrors) are known. The challenge in computing an unknown distance from multiple images using reflecting surfaces is called catadioptric stereo vision. In J. Gluckman and S. K. Nayar: Catadioptric Stereo Using Planar Mirrors (International Journal on Computer Vision, 44(1), pp. 65-79, August 2001), the basic theory of catadioptric stereo image generation is described in detail. In this paper an image capturing setup including one camera and one planar mirror is introduced with a known relative position of the camera and the mirror, and hence calibration is not needed This method results in a volumetric 3D representation of an object in the real camera's view.
In the paper of Hu et al. ‘Multiple-view 3-D Reconstruction Using a Mirror’ (ftp://ftp.cs.rochester.edu/pub/papers/robotics/05.tr863.Multiple-view_3-d_reconstruction_using_a_mirror.pdf) a stationary camera and a planar mirror are used for multiple-view three dimensional object reconstruction. The distance between the mirror and the camera is obtained by a single object point and a pair of points of the mirrored view of the object. The mirror image of the camera is searched in the captured images and then the epipoles of the virtual camera are used to determine the spatial relationship between the virtual camera and the real camera. This method, however, cannot be used for 3D object reconstruction if the real object is not visible in the captured images.
The paper of Kumar et al., ‘Simple calibration of non-overlapping cameras with a mirror’ (http://frahm.web.unc.edu/files/2014/01/Simple-Calibration-of-Non-overlapping-Cameras-with-a-Mirror.pdf), introduces a calibration method for set of cameras. Although this method also uses the mirror images of the cameras, it does not use the images of the real object, and therefore at least five images are required in order to recover the real camera position and orientation.
Calibrating a stereo (or multi-view) camera system is a difficult task. In general, it requires to find several corresponding points in the captured images, and then to solve a non-linear optimization problem with six to eight parameters (depending on whether or not the focal lengths of the cameras are known). In our proposed method, calibration can be obtained by reducing the aforementioned optimization problem to two independent, much simpler optimization problems, each having three or four parameters (depending on whether or not the focal lengths are known). Due to this decomposition of a more complex computation into two simpler computations, the method of the invention is faster, more reliable and more robust than the prior art calibration methods.
The key idea of the calibration methods of the present invention is that by using the mirror view of the real camera along with multiple (different) views of an object in one or more captured images, the 3D coordinates of a point in the real space with respect to the mirror coordinate system can be easily determined. Additionally, by selecting two spatial points which both appear in the one or more captured images, the real distance between the two points can be determined on the basis of their corresponding image points.